Conventional processes for colorization of grayscale images and re-coloring of color images can produce visual artifacts in the resulting image. For example, conventional colorizing processes, particularly automatic color image processing/mapping methods, such as color transfer and color harmonization, can produce color inconsistency in areas where the color and/or luminance values change smoothly in the original image. Thus, strong artificial color edges that do not exist in the original image may be created in the resulting image. In addition, other types of image processing that do not modify color directly can also produce visual artifacts that affect the color of an image.